Node Aggregation for Distributed Inference in Bayesian Networks

This study describes a general framework and several algorithms for reducing Bayesian networks wi th loops (i.e., undirected cycles) into equivalent networks which are singly connected. The purpose of this conversion is to take advantage of a distributed inference algorithm |6|. The framework and algorithms center around one basic operation, node aggregation. In this operation, a cluster of nodes in a network is replaced with a single node without changing the underlying jo int distribution of the network. The framework for us ing this operation includes a node aggregation theorem which describes whether a cluster of nodes can be combined, and a complexity analysis which estimates the computational require ments for the resulting networks. The algorithms described include a heuristic search algori thm which finds the set of node aggregations that makes a network singly connected and allows inference to execute in minimum time, and a "graph-directed" algorithm which is guaranteed to find a feasible but not necessary optimal solution and with less computation than the search algori thm. 1 I n t r o d u c t i o n This study describes a general framework and several algorithms which use that framework for converting Bayesian networks wi th loops (i.e., undirected cycles) into equivalent singly-connected networks. The purpose of this conversion is to take advantage of a distributed inference algorithm [6]. The framework and algorithms center around one basic operation, node aggregation. In this operation, a cluster of nodes in a network is replaced with a single node without changing the underlying joint distr ibut ion of the network. Like its predecessor technology, decision/risk tree technology, Bayesian Networks (a.k.a. influence diagrams) [3], is a technology for representing and making inferences about beliefs and decisions. A probabilistic Bayesian Network is a directed, acyclic graph (DAG) in which the nodes represent random variables, and the arcs between the nodes represent possible probabilistic dependence between the variables. A network as a whole represents the jo int probability distribution between the random variables. The representation has proved to be an improvement over the older tree technologies for several reasons including increased functionality, compactness, and intuitiveness to users. While a fast distributed inference algorithm exists for singly-connected networks, it has been proved [ l ] that no algorithm can be efficient on all Bayesian networks with loops. However, there appears to be much room for expanding the set of graph topologies which can be addressed in a computationally efficient manner. Other than the node aggregation approach presented in this study, several other approaches have been proposed to address the inference problem for arbitrarily-connected networks. These approaches include: conditioning |7|, cliques [4], using the influence diagram operations such as link reversal and node removal [8], and stochastic sim ulation [7]. In this paper, we have chosen the node aggregation method to handle the inference problem in an arbitrary network. For all node aggregation methods, the first and defining step is to reduce the graph using node aggregation into a singly connected graph. This step needs only occur once. In the second step, the distributed infer ence algorithm is applied to the reduced graph to calculate the posterior distributions of each node. Since the aggregated nodes in the reduced graph may consist of more than one original node, the third step calculates the posteriors of the original nodes by marginalizing the posterior probabilities of the aggregated nodes. The paper is organized as follows. Section 2 describes the definitions and theorems which make up the framework. Of principal interest is a node combinability theorem which determines if a set of nodes can be aggregated. The effects of a node aggregation on a graph are then described. A computational complexity measure is also presented in Section 2. Given a graph's topology and the state space size for each node in the graph, this measure calculates the approximate computation time required for each update in inference. In Section 3, an A* search algorithm to find the optimal node aggregation parti t ion is developed based on the performance criterion obtained in Section 2. This algorithm utilizes pruning techniques that can substantially reduce the search space and the optimal solution is guaranteed to be retained. A much